<12) 



UK Patent Application „»GB ,,,,2 331 880 „3,A 



(21) Application No 9825788.4 

(22) Date of Filing 26.11.1998 



(43) Date of A Publication 02.06.1999 



(30) Priority Data 
(31) 979838 



(32) 26.11.1997 (33) US 



(71) Applicant(s) 

California Institute of Technology 
(Incorporated in USA - California) 
1201 E California Boulevard, Pasadena, 
California 91125, United States of America 

(72) Inventor(s) 

Rodney M Goodman 
Karen Emerson 
Jeffrey Dickson 

(74) Agent and/or Address for Service 
Eric Potter Clarkson 

Park View House, 58 The RopewaJk, NOTTINGHAM, 
NG1 5DD, United Kingdom 



(51) INT CL 6 





H04H 9/00 


(52) 


UK CI (Edition Q) 




H3Q OAR Q200 


(56) 


Documents Cited 




US 4720873 A 


(58) 


Field of Search 



UK CL (Edition Q ) H3Q OAR 
INT CL 6 H04H 9/00 
ONUNE:WPI,EPODOC, PAJ 



(54) Abstract Title 

Television advertising automated billing system 

(57) An arrangement is described for determining whether an advertisement has been broadcast and if 
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which code may include information associated with the content. Various security measures to prevent 

r!Zw r l n9 t0 ^l e>/e T de ! ectfon b Y aromatic detection equipment in receivers, which would permit 
recording without the advertisements, are described. 



102- 



Ad 



141- 



HE 



X 



A0TAG 
Encoder 



160 



XL 



^42 



TVS 



BOOKING 
DATABASE 



-164 



BILLING 
COMPUTER 



^162 



ADTAG DECODER 




14a 
146 


ADTAG 1 
RECEIVER | 








CODE r- 
STRIPPER i- 








\~iso 


REAL 1 
TIME CODEl 




148 y 



TRANSMITTER 



DATA 
TRANSFER 



-152 



FJG. 1B 



CD 

ro 

CO 
CO 



oo 
oo 



At least one drawing originally filed was informal and the print reproduced here 



is taken from a later filed formal copy. 



wsnnrin- ,no ->■»■>. son, , . 
BNSOOCID: <GB 2331880A9_L» 



2331880 

- 1 * 

TELEVISION ADVERTISING AUTOMATED BILLING SYSTEM 
FIELD OF THE INVENTION 
The present invention relates to a media 
advertising automatic billing system which automatically 
5 and securely maintains an exact tracking of 

advertisements which are played and produces billing 
information for those advertisements. 

BACKGROUND OF THE INVENTION 
Commercial advertising ("commercials") create the 
10 revenues that sustain the content provider's operation. 
Commercials provide a host of special problems based on 
the nature of the media. 

Specifically, media such as television and radio 
sell specific time slots for playing of the 
15 advertisement. The content provider, e.g., the 
television station, sells a time slot in which a 
commercial will be played. 

This arrangement creates special considerations. 
The station is selling time. Once that time passes, the 
20 value of the service goes to zero. Similarly, after the 
commercial plays, there is no way of unplaying it. This 
compares with other sellers of commodities, who often 
have the option to repossess the chattel being sold. 

Because of these considerations, most stations 
2 5 require that payment for booking be entirely in advance, 
except for the best, i.e, the most creditworthy accounts. 

The most typical advertisement includes regular 
television commercials, typically thirty to sixty seconds 
in length, which are aired during breaks in commercial 
30 television programs. The so-called infomercial is also a 
television commercial advertising, but are typically much 
longer in length, e.g. between four minutes and one hour. 
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With the exception of certain public service- 
oriented advertising, all commercials are paid for by a 
sponsor. The billing is done on a per-piece basis, 
based on the time and likely audience for the commercial . 
5 This pricing is a multi variable determination. For 

example, a one-minute commercial aired during the Super 
Bowl is considerably more expensive than a one -minute 
commercial aired at 2:00 A.M. Therefore, it is extremely 
important that the time when the commercial airs is the 
10 same time as what was paid for. However, it is not 
always a routine matter for the station to determine 
this. 

A nationwide television commercial is often 
received by the television station, e.g., via its 
15 satellite transponder hook-up. A "nationwide" television 
commercial is often included as part of the nationwide 
program. In this case, the central network provider 
provides both the television program and some of the 
commercials to be aired everywhere. However, other 

2 0 breaks within the commercial television program may be 

filled by "local" commercials, those that are aired only 
locally. These local commercials are typically produced 
locally, recorded and stored at the television station. 
Nationwide programming time schedules for the 
25 affiliates are sent to local television affiliates from 
the network. This includes a schedule describing open 
times for local advertisement station identification and 
other time slots that the station can sell locally. Each 
half hour slot generally has two programming slots of 12 

3 0 minutes duration with commercial slots in between. The 

national schedules also often leave half hour time slots 
open for infomercials or public access programming. 
Television guides often show these slots as "paid 
programming" . 
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Depending on various circumstances, the local 
television station may have the option to override the 
national commercial. At times, the local station 
overrides the national commercial even when such an 
5 override is not authorized. This provides a significant 
problem of tracking which commercials have been shown and 
when . 

The advertiser may pay in advance for a half hour 
to be aired at 7:30 P.M., but the commercial actually 
10 airs at 10:00 P.M. These times may have radically 

different prices. The price of a commercial aired at 
9:02 P.M., which is after the show that aired in the 8:30 
to 9 time slot can even be significantly different from 
the price of a commercial aired at 8:58 P.M., which is 
15 during that time slot. The advertisers are 

understandably, therefore, extremely interested in 
knowing the precise time of airing. However, the varied 
nature of the advertising system, and especially the 
television system often makes it difficult to determine 
20 precisely when the commercial will be aired. 

Typically, the advertising agency acts as a 
intermediary between the actual advertisers and the 
television station. The advertiser wants their 
advertisement to air at certain times on certain channels 
25 and in certain events. For example, a toy company might 
want their advertisement to air during the Sunday morning 
cartoons. Airing the advertisement during Sunday 
football would not reach the desired audience. 

Special prices are often negotiated based on 
30 whether the advertisement is in prime time, off prime 
time, holidays, weekends, special events or the like. 

At some time before the actual airing of the 
broadcast, the station produces an actual program 
schedule which show the programming segments it will play 
35 or receive, as well as all commercials, infomercials and 
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station identifications. Each of the latter are stored 
on tape which are pulled from the video library by a 
station engineer. . Commercials and inf omercials are 
usually manually played by the station engineer who 
5 initiates the proper tape based on a schedule in a 

specified time slot. The engineer keeps manual records 
of what has actually been done and the billing of 
commercial time is carried out based on these records. 

The actually-played commercials are often sampled 

10 by third party organizations who employ people to watch 
and keep track of television commercials which are 
played. These people, however, cannot watch every single 
television commercial, and therefore only act as a 
sampling mechanism to determine a percentage of correct 

15 reporting by the television station and advertising 
agency . 

One other means of verification of digitizes one 
or several frames of the commercial. Then the physical 
airing is taped and compared to the frames. This still 
20 requires a television to receive the broadcast and a 
physical comparisons of the frames. This method is 
inaccurate and again based on statistical sampling. 

SUMMARY OF THE INVENTION 

2 5 Recent studies have suggested that perhaps as many 

as fifty percent of the reported commercials are not 
played in their proper time. This is often because of 
the great demands that are placed on the station 
engineers. The station engineers are often simply too 

30 busy to keep accurate reports of the commercials in view 
of the many different demands that are placed on them. 
Moreover, since the scheduling process is largely fluid 
in nature, the reports made by these engineers are 
crucial. The billing and accounting is done based on 
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these reports, and their accuracy is, as described above, 
difficult to ascertain. 

In addition, moreover, the manual billing system 
described above requires a crucial part of the billing to 
5 be manually entered. This system has never been 
satisfactorily automated. 

Added to this is the problems obtained from the 
nature of the system in which the commercial can't be 
taken back once it has been played. Hence, the stations 

10 have no effective means of repossession. 

The inventors of the present invention recognized 
all of these problems and realized that a real-time 
automated system for tracking the actual commercials that 
were aired would effectively be a win-win situation: for 

15 not only the advertisers who would get an exact 

accounting of v/hen their commercial was aired, but also 
the television stations who would know exactly when the 
commercial was aired and would secure an automated means 
for determining the real percentage of scheduling 

2 0 anomalies. 

The current system of payments for commercials by 
the advertiser usually requires payment prior to the 
airing of the commercial. In addition, the TV station is 
often required to sign an affidavit confirming the actual 
25 airing if so required by the advertiser. An objective of 
the present system is to have a system that pays for the 
commercial on a "real-time* basis using an automated 
clearinghouse system. The objectives include: 

i. The TV Station gets a means to receive 

30 payment for their services - which is why 

advance payment is the standard norm of 
payment terms; 

ii. The advertising agency gets a means to 
receive payment for their services - a 

35 commission for the actual airing of time 



BNSOOCID: <GB 2331880A9J_> 



) 



booked by the agency; and 
iii. The advertiser (Sponsor) gets a means to 

insure airing of a commercial at the agreed- 
upon time and price. 

5 Each entity wants accurate accounting of the 

commercial in addition to a more efficient, less 
cumbersome process of the payment transaction with 
reduced collection cost. This system describes a 
clearinghouse system to meet all of these objectives. 

10 In view of the above, the inventors have provided a 

system which marks the commercials in a way that ensures 
secure and unambiguous identification of the subject 
matter. Each commercial is played by the television 
station including a secure identification. That 

15 identification is received, date and time confirmed, and 
verified. The information is sent to a billing computer. 

The security is enhanced in this system. One 
embodiment uses a cryptographic system. Another uses a 
system which restricts access to the hardware. 

20 A cryptographic system can be used to ensure that 

the information has not been tampered or faked. This 
provides the billing computer with a secure, computer- 
manipulable record indicating when the commercial was 
aired. The billing is done automatically by a computer 

2 5 program based on the data received from the actual airing 

of commercials . 

The inventors also noticed a number of problems 
which could exist in such a system and have provided 
solutions to these problems also. One problem comes from 

3 0 the marking of commercials itself- Many have tried to 

analyze the content of commercials, so that such 
commercials could be automatically identified by an 
electronic circuit- Such an electronic circuit would 
then presumably either blank the commercials or turn off 
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the video recording device so that the show could be 
watched in its entirety without commercials. Naturally, 
the advertisers are very hostile to the existence of such 
a system, since this would defeat the effect of their 
5 paid-for commercial time. A code on each commercial 
would be a sure way of detecting such a commercial. 
Therefore, the system preferably includes a way to avoid 
detection of the commercial. One way is by removal of 
the code prior to airing. 
10 Another problem is the possibility of dishonest 

television stations or billers. A secure system requires 
that non authorized persons be prevented from tampering 
with the detection or improperly or falsely providing the 
detection. In view of the above, the inventors have 
15 devised embodiments which include special encryption 
techniques are used which make it difficult for an 
unauthorized party to properly code the commercial in a 
way that would fool the billing computer. 

Another system correlates content of the commercial 
20 with the billing system to avoid the possibility of 

simply copying a code from one commercial on to another. 

Another system adds some measure which prevents 
using this system to detect whether program content is a 
commercial . 

25 The present system, titled the "ADTAG" system, uses 

a clearinghouse concept. Once the billing information is 
collected via the tracking system, a settlement between 
the advertiser, the advertising agency, and the TV or 
radio station is made via electronic means such as a 

30 modem or the Internet. This settlement is an accounting 
of the transaction and payment for the transaction 
between the three entities- again via electronic means. 

The ADTAG clearinghouse system includes a 
verification portion which verifies in real time that a 

3 5 commercial has actually been aired, along with 
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verification of the time when the airing occurred. This 
information is used to authorize payment if all 
information matches or to initiate an error resolution 
procedure if the information does not match. 



5 BRIEF DESCRIPTION OF THE DRAWING S 

All of these will be described in detail with reference 
to the accompanying drawings wherein 

Fig. 1A shows a block diagram of the clearinghouse 
of the present invention; 
10 Fig. IB shows a basic block diagram of the overall 

system of the present invention; 

Fig. 1C shows an alterative block diagram of a 
di f f erent embodiment ; 

Fig. 2 shows a tape tagger; 
15 Fig. 3 shows a diagram of the teletext- text packet 

system ; 

Fig. 4 shows one possible encryption system for 
encoding; 

Figs. 5A and 5B show the schematic of the preferred 
2 0 system of the present invention; 

Fig. 6 shows an encoder flow diagram; 
Fig. 7 shows a television monitor system block 
diagram; 

Fig. 8 shows a decoder flow diagram; and 
25 Fig. 9 shows a flowchart of the billing and 

verification done by the ADTAG system. 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The present specification describes billing for an 
advertisement which is broadcast over a medium. The 
30 medium could be of any type. Television is described as 
the preferred embodiment of the medium. 

video sent over a channel is often transmitted using 
the teletext standard, which is a packet transmission 
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method for transmitting video. Th video is formed of a 
number of pixels arranged in lines. The lines are used 
to transmit the digital information. 

Each line is transmitted from the left side of the 
5 screen to the right side of the screen. The cursor that 
writes the information thereafter needs to move from the 
right side of the screen back to the left side of the 
screen. The display is blanked during this time of 
retrace. This time is referred to as the vertical 
10 blanking interval or W VBI" . Information can be stored in 
these times of retrace. 

The NTSC video standard uses 52 5 lines with VBI 
lines in lines 6 through 21 and lines 258 through 273. 
Some of these lines are already in use for other 
15 functions. For example, VBI line 21 includes codes that 
are indicatove of closed captions. 

Fig. 3 shows the contents of the teletext packet. 
The control bytes 300 include header information that 
identifies the contents of the user data that includes 
20 the ADTAG packet. This allows the system to mark the 

data as proprietary to avoid confusion of other teletext 
receivers by using a unique code. The user bytes are 
encoded in order to ensure reliable transmission. The 
present system uses one or more VBIs per full frame of 

2 5 video. The raw data frame without forward error control 

includes 28 bytes per packet (8 bits per byte, sixty 
packets per second = 1340 bits per second) requiring a 
data transmission speed of 13.44 kbs. 

A preferred forward error correction system includes 

3 0 the single error correcting double-error detecting 

Hamming code. Many other codes could be used. 

This allows a true reliable data rate of 6.72 
kilobits per second. Therefore, for example, in the 
space of time taken up a 30 second television slot, 
35 approximately 200,000 bits of reliable information can be 
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delivered. Standard DES with 64, 128, or 256 bit keys 
could be used. Preferably, the system also uses some 
correlation technique that correlates the code with the 
commercial in a way that prevents simply copying the code 
5 from one commercial to another. 

The clearinghouse system is illustrated in Fig, 1A. 
To use the system, an advertiser 180 usually first 
obtains a line of credit for advertising from a financial 
institution 182. This can be considered roughly 

10 analogous to a VISA card. The financial institution 

assigns an account number and credit limit. This account 
number will be used for charging later advertisements. 
The user then joins the ADTAG clearinghouse 186. 
The ad agency 184 then books a commercial by obtaining 

15 the advertiser's account code along with other 

information about the advertisement. The booking 
information may include: 
BOOKING INFORMATION 

Agreed upon air date and time, duration of commercial 
20 Advertiser account code 
TV station account code 
Tape name and production date 
Ad agency account code 
Agreed upon pricing and commission 

25 This information is usually obtained by the ad 

agency. However, larger advertisers can have their own 
in-house booking departments that could sign on to ADTAG 
directly. ADTAG then takes this information and develops 
reports for each entity based on the service agreement 

30 signed with ADTAG. Reports will be for that entity only 
- i.e., ad agency #1 can only get information related to 
ad agency #1 and not for any other entity. Reports can 
be generated including: 
Conflicts in scheduled time 
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Daily, weekly, monthly booking report for planning 
purposes 

TV stations booked 
Booking by advertiser. 

The ad agency also joins the ADTAG clearinghouse 
system and agrees to abide by its program guidelines. 
The ad agency books a commercial or time slots with the 
ADTAG clearinghouse for its customer. The agency enters 
its account code, the customer accounts code, pricing for 
the commercial and the agency's commission code into the 
clearinghouse data base. At this point a reserve is 
entered onto the advertiser's credit line - similar to 
the way in which car rental firms and hotels put a 
reserve on a credit card when you rent a car or room. 
15 This reserve is removed by ADTAG when actual airing data 
is received or the air date time has expired. 

ADTAG provides reports and information on a real 
time basis back to the ad agency, TV Station, and 
advertiser, based on the level of service selected by 
2 0 each entity. 

When a commercial is played. ADTAG also decides if 
the commercial has been played at the precise same time 
as what was booked. If so, ADTAG "verifies" the playing, 
and "authorizes" payment from the bank. if not. the 

25 system initiates error resolution procedures. 

An advertisement is made by digitally encoding 
certain information onto the tape which will be used to 
air the advertisement. Tape encoding will be provided by 
ADTAG or an ADTAG endorsed duplication house. The 

30 encoding is done on the master tape of the commercial 
production. In that way, any tape duplicated off the 
master tape will include all the proper coding. m 
addition, ADTAG uses some kind of verification of 
security. This can be an encrypted code on the tape, 
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and/or requiring secure premises and secure handling of 
the tape. The tape is then sent back to the advertiser 
(sponsor) or to whomever the sponsor authorizes. This is 
the first part of the security system. This information 
5 preferably includes, for example: 

• the advertiser's ADTAG account code: 

• The name of the tape 

• Date the tape was produced 

• Last date the tape should be used 
10 • Certain copyright information 

• Any other information about the tape which may be 
useful , 

The information is preferably digitally encoded in 
some way associated with the advertisement that is 

15 actually playing during the advertisement. 

The ADTAG code is included on the tape by a secure 
encoder. This can be done as follows: 

a) The preferred technique uses ADTAG in a secure 
premises. The master tape is sent to ADTAG who adds the 

20 code. The master tape is returned including the 

duplicating code, and sent to the duplication house and 
then on to the TV stations. This allows keeping the 
encoder only at secured premises. Powerful one- time-pad 
codes can be used in this case, e.g., where only the 

2 5 ADTAG computer knows the key. 

A possible disadvantage that this system requires 
one extra recording and this may reduce the quality of 
the master. An all-digital system can be used to 
minimize any such degradation. 

30 b) ADTAG -produced encoder boxes that are sent to 

any person who wishes to put ADTAG codes on a master 
tape. The encoders use an encryption scheme. This 
alleviates the problem of requiring an extra recording. 
But this has a possible problem that the encoder boxes 
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could be subverted if the code is broken. 

The preferred technique uses the digital encoding of 
the information into the vertical blanking intervals in a 
secure manner using a key encryption system. This system 
5 that is used needs to be secure and able to detect 
attempts to tamper with the code. 

c) An alternate technique encodes information as 
part of the closed captioning part of the commercial. 
This can be readable information, e.g., the first line of 
10 text of closed captioning can be used to identify the 
commercial . 

The security in this case comes from the literally 
thousands of people who will see the captions. 

The tape is sent to the television station which 
15 sets it up for play matched to the station production 
schedule . 

The station also may include an ADTAG decoder 
receiving the actual broadcast information. The ADTAG 
decoder recognizes and/or strips the ADTAG code. 

20 Three embodiments of the ADTAG decoder are 

described. All require an ADTAG decoder which will decode 
the ADTAG information and accumulate the billing data for 
downloading to ADTAG. The ADTAG decoders are preferably 
tamper proof boxes, which are, for example, potted in 

2 5 epoxy, and/or formed with an electronic jumper which 
automatically erases memory contents when the box is 
opened. Each have unique ID's (keys) stored in areas of 
nonexternally- readable ROM blown into the microprocessor. 
The code can be used in three ways: 

30 l) The ADTAG code is broadcast with the commercial. 

This has the possible disadvantage noted above that the 
commercial may be identifiable. 

A major advantage of this system is that the ADTAG 
decoder does not reside at the TV station, but can reside 

35 at a remote (secure) ADTAG site 184 that is monitoring 
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the broadcasts of the TV station. Only those 
advertisements which are actually broadcast, therefore, 
are detected* 

Certain countermeasures described below could be 
5 used to avoid detection of the commercials. 

If the code uses a part of the closed captioning, 
this is also unlikely to be usable as a means to detect 
codes, since the closed captioning occurs on many 
different types and areas of programs . 
10 2) Alternatively, the ADTAG code is stripped off 

and detected. The VBI lines used are returned to black 
before the video is broadcast out. 

Here, the ADTAG decoder resides at the TV station, 
and in series with the live feed. Hence this is a 
15 potential degrader of signal quality. It is also subject 
to "spoofing" by technicians at the station. Hence, one 
could feed a completely false video stream to the ADTAG 
box and a completely separate video stream to broadcast. 

This could be alleviated by having the ADTAG box 
20 monitor the transmission of the TV station by, i.e., 

including a receiver and matching the claimed live feed 
to the received live feed. It is difficult, however, to 
make this system completely secure. 

Advantage: nothing is broadcast in the VBI so 

2 5 commercial detection in this way becomes impossible. 

3) Another embodiment uses the ADTAG box to add the 
code to ALL lines including programming, and all lines 
are broadcast with a code which only ADTAG can decode. 
Hence if the ADTAG box detects this is an ADTAG coded 
30 commercial, it allows the code to pass through after 
date/ time modification. If it detects no ADTAG code it 
adds a random code to the line. The disadvantage is that 
the ADTAG box is in series with the live video feed as in 
b. above. There are two major advantages: 1) as in the 

3 5 above, the ADTAG monitor can be at a remote site and 
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hence secure. 2) The commercial cannot be distinguished 
from regular programming. 

Another possible problem is spoofing by a malicious 
TV station. That is simply copying the valid ADTAG codes 
5 and splicing them onto some other programming, or other 
commercial tape. They could not read the content of the 
tape - but copying is easy. 

But why would the TV station want to do this? This 
could make it possible to take the ADTAG codes from a 3- 
10 minute COKE commercial and splice them onto a 3 -minute 
PEPSI commercial. Thus ADTAG would allow COKE to pay the 
bill, and the TV station would also collect from PEPSI . 

According to the present invention, counterfeit 
prevention techniques are used. A first technique links 
15 the actual picture content to the ADTAG code. 

A technique of characterizing pictures in current 
programming can be used. For example, one method 
extracts a quantity such as the average picture 
brightness level over the last X frames and adds that to 
2 0 the ADTAG code. 

For example, this system uses a capacitor or 
integrator to compute the average brightness level over a 
line. That analog value is converted to digital and 
added to the encrypted code, 
25 A sigma-delta ADC could do this all in one 

operation. Simple pasting of the code onto a different 
video would then only work for a video with the same 
statistics as the original. The actual method does not 
need to be terribly robust. The A/D need only be at most 
30 3 bits, maybe only 1 bit, e.g., a comparator which 
compares the average brightness to a threshold. But 
taken over ALL the lines there is a lot of information. 
The confidence factor can be changed by increasing the 
accuracy. Other characteristics could also or 
35 alternatively be samples, such as audio, or other more 
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complicated aspects of the picture, including numbers of 
objects, histogram of colors in the image. 

The close captioning embodiment uses identifying 
information in the existing close captioning lines. Thus, 
5 the ADTAG tape would have close caption information, 
plainly readable, e.g., this is a COKE commercial 
AD#23456 3 minutes .... END of AD#23456, etc., etc. which 
would be broadcast out in the clear. 

A disadvantage is that anyone can read the info and 

10 one can tell the commercial slots. But it has the big 
advantage that if many people are watching the close 
caption data, it becomes difficult to change it without 
detection. If millions of people see a Wally's painting 
advert on the picture, while COKE is on the close caption 

15 data, the phones will ring. Thus, third party 
verification is possible and spoofing becomes 
impractical . 

Yet another technique uses existing cryptographic 
techniques to digitally sign the code. 

20 In addition, the ADTAG decoder 141 verifies that the 

tape was actually broadcast as data is being collected 
and stripped. Verification of the broadcast is included 
in the stripped data sent to ADTAG database computer. An 
alternative to stripping the codes prior to broadcast is 

25 to leave the codes on with the broadcast and allow a 

third party verification of the broadcast with the codes. 

When a commercial is aired with associated ADTAG 
information, the decoder 141 gathers the information from 
the tape using code stripper 146, and adds to that 

30 information a date and time stamp at 148, booking code, 
ADTAG account code security code and time length of the 
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commercial. Therefore, this information stripped from 
the ADTAG decoder attached to the live broadcast feed 
line of the TV station includes: 

ADTAG security code (i.e., ADTAG encoded on the master 
5 tape) 

TV station code 
Broadcast verification code 
Tape name 
Advertiser code 
10 Tape production stale date 

Date, time and length of airing 

This combined information is sent to the billing computer 
162 . 

The added information is preferably stripped prior 
15 to actual broadcast of the live feed as shown in Fig. IB. 
This is done to avoid the possibility, as described 
above, of providing an icon from which a commercial can 
be automatically detected. It is important, however, 
that the information be received while it is actually 

2 0 being transmitted, to ensure real time reception of the 

advertisement. The central system then matches the 
following information to its booking system, and provides 
automatic billing. 

If a match is accurate, then the billing system 
25 batches the transaction. Later, the system initiates via 
either modem or Internet: 

A banking transaction to remove the reserve from the 
credit line; 

Another banking transmission to add the total amount 

3 0 due to the advertiser; 

Another modem transaction to provide advertising 
agency's cut of this transaction. 
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An alternative scenario is where there is a problem 
with the actual airing or with the airing not matching 
the booking in the system. 

All of this functional information will be described 
5 in further detail with reference to the remainder of the 
disclosure . 

The basic block diagram of the overall system is 
shown in Fig. IB. This block diagram shows the system of 
stripping the codes at the broadcasting station, shown 

10 here as a TV station, prior to transmission. The three 
major components of the system include production of the 
advertisement at 100, airing of the advertisement at 140, 
and billing of the advertisement at 160 using the ADTAG 
clearinghouse shown in Fig. 1A. 

15 The advertisement is formed by advertising material 

102. The advertising material is encoded with special 
ADTAG information from ADTAG encoder 104. The combined 
information is placed onto an advertising media 106 which 
is for example a video tape. The video tape is then used 

20 by the television station at 140 in order to transmit the 
information. 

The transmitted information from TV station 14 0 is 
received by an ADTAG receiver 144 which strips the ADTAG 
code at 146. A real-time clock 148 is combined with the 
25 stripped codes to produce a composite stream 150 

including both the codes and the real-time information. 
This information is transferred by a data transfer 
mechanism 152, for example a modem or some other TCP/IP 
mechanism. 

30 The video information with the code stripped off is 

transmitted by transmitter 154 in the usual way. 

The billing information is transferred to billing 
computer 162 at area 160. The billing computer compares 
the incoming data with information from its booking 

35 database 164 . This information is then used for 
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appropriate billing. 

The Fig. IB block diagram of the system shows the 
basic functionality of the system. A number of 
improvements and modifications are contemplated. 
5 Initially, the need for security is paramount in this 
system. Use of appropriate encryption is crucial for 
ensuring the security of the system. The encryption must 
be sufficiently robust to avoid falsification and detect 
tampering. All of the ADTAG components, including the 
10 ADTAG encoder 104, receiver and code stripper 146 and 
billing computer 162 preferably include encryption and 
decryption circuitry including the capability of 
determining any attempt at alteration of the code. 

Although this system is described for use with 
15 television, it is equally applicable for use with any 

other communication system, including satellites, closed 
circuit television, Internet, royalty tracking, radio, or 
any other means. Also, while the preferred embodiment 
describes certain identifying information being placed in 
20 vertical blanking intervals of the video, and later 

removed, it should be understood that any form of data 
encoding which does not interfere with the viewer's 
watching of the actual commercial could alternately be 
used. A reverse stripping of the codes, where the codes 
25 are broadcasted and stripped by the radio listener and 

royalty trackers could also be used. other schemes would 
include putting the information in the picture itself 
using steganographic methods. This is currently being 
done in still images by Digimarc™. These systems adjust 
30 portions of the image that cannot be seen with the naked 
eye, in a way that encodes the information. 

A particularly preferred embodiment described 
includes information in the vertical blanking intervals 
along with the ADTAG codes of a number of different 
35 frames. These items of information need to agree and be 
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correct in order to verify the information. 

Fig. IB shows the block diagram for embodiment B 
where the ADTAG is stripped off and detected before the 
video is broadcast. Fig. 1C shows an alternative block 
5 diagram for embodiment A where the ADTAG code is received 
a remote ADTAG site 12 0 which includes an ADTAG decoder 
stripping the codes and transferring them to the billing 
computer system 160. 

Embodiment C also uses the same block diagram of 
10 Fig. 1C. 

A diagram of the tape tag encoder system is shown in 
Fig. 2. The tape tag encoder forms the ADTAG encoder 
system 104 that is used to tag video material as it is 
placed on storage media, e.g. tape 106, prior to 

15 broadcasting. This system is used for example at 

production houses, which produce the advertiser's tapes 
that are sent to the broadcaster. 

Controlling console 200 allows input of certain data 
to be recorded in the tag. This data includes, for 

2 0 example, information about the advertisement including 
but not limited video information indicating the 
advertisement. The video 202 from video source 200 is 
tagged using a tape tagger element 204. The information, 
once tagged is recorded by a video tape recorder 206. 

2 5 The tape tagger 2 04 operates as described with reference 
to the schematic of Figs. 5A and 5B to modify the 
teletext -encoded video. 

As described above, the preferred mode operates to 
encode the required information into the vertical 

30 blanking interval of the video signal, e.g., the NTSC 
signal. Of course, it is also possible to encode the 
information into the video signal in other ways. The 
system that is described in the following explains how 
the information is encoded on the video signal. 
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The schematic of Figs. 5A and 56 shows a schematic 
which carries out the decoding and encoding. The circuit 
is encased in potting epoxy for physical security. Also, 
keys can be held in secure areas of RAM that are not 
5 externally readable and are only readable in the 

microprocessor internals. The video signal is received on 
the video input port 500. The signal is modified by the 
elements of the circuit. The modified composite video 
signal is output through video output port 502. Almost 

10 any line can be selected, e.g., line 7. 

The circuit in Figs. 5A and 5B operates both as an 
encoder and a decoder, based on a software switch. The 
circuit operates as an encoder to replace a line of video 
in the vertical blanking interval of an NTSC video signal 

15 with a new signal that includes the desired encoded data. 
Fig. 5B shows the serial port 550 which is connected to a 
controlling computer. This controlling computer 
communicates information, including operating commands 
which are dated and encoded. Data that has been 

2 0 recovered can be sent to the computer, or sent over the 
modem line to a central billing computer. 

Depending on the amount of data that is transmitted, 
much of the video signal may be simply passed by the 
circuit from the video input to the video output. The 

25 signal that is unchanged passes through video multiplexer 
504 to buffer 506 and is finally output by 502. The 
portions of the video signal which are altered are 
treated differently. 

The video signal is also sent to video -in node 552 

30 in Fig. 5B. The video-in node 552 is coupled to a PIC 
microprocessor 554 which carries out the various 
functions of the circuit. This includes monitoring the 
timing information from an LM 1881 integrated sync 
separation chip 556. This circuit indicates occurrences 

35 of vertical sync, horizontal sync and burst in the NTSC 
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video. The video processor 556 also indicates whether 
the video currently represents even or odd field of the 
NTSC frame for use in selecting which field is encoded. 
Processor 554 monitors the sync pulses in order to 
5 locate the desired portion of the video blanking 

interval . Each sync pulse represents the start of a new 
video frame. Therefore, a vertical sync event is taken as 
a reset for the line count. Each horizontal sync pulse 
indicates the start of another line of video. The program 

10 uses these pulses to count lines until reaching the 

desired line. This desired line, for example line 7, is 
the portion of the frame which this circuit will encode 
the user data. This portion is called the VBI window. 
Almost any line can be selected, e.g., line 7. Some 

15 lines are used for sending broadcast level information, 

and the V-chip uses certain lines. We need to avoid these 
lines. In fact the ADTAG encoder could look for tt the 
first unused line" and simply use that. It would also 
change which lines it uses. 

2 0 The circuit operates according to the flowchart of 

Fig. 6 to encode information on the video signal. First, 
processor 554 generates a SampleBlank signal which causes 
a blank level sampling circuit (blank) to sample the 
blank level . Processor 554 also sets the program VBI 

25 signal to block the original video signal from passing 
through video multiplexer U8 during this time. During 
the time that the program/VBI signal is active, the 
replacement signal passes through video multiplexer U8 . 

User data is encoded at step 606 in the flow diagram 

30 by generating a representation of the user data on the 
VBI data line. The preferred mode described herein 
encodes a digital "0" as the equivalent of a white dot in 
the video signal and encodes a 0 as a black dot. It is 
also possible to encode the information in analog form by 

35 changing the color levels for example. Processor 554 



BNSDOCID: <GB 2331880A9_t_> 



- 23 - 

interfaces to the serial port via serial port driver 560 
using a slave microprocessor 562 to carry out appropriate 
formatting of information including control commands and 
user data. Serial port driver adjust the signal level 
5 from the PIC level used by the PIC microprocessor to a RS 
232 communication level. The slave processor 562 can be 
for example a PCI processor which generates the sequences 
required to drive the display and interface to the ports. 
The SampleBlank signal described above is fed to at 

10 least two circuits. SampleBlank is fed to transistor 510 
along with the associated circuit 512 including an 
operational amplifier. When SampleBlank is asserted, the 
transistor 510 conducts causing capacitor 514 to charge 
and discharge in equilibrium with the input video signal. 

15 When SampleBlank is later deasserted, C2 sees high 

impedances on the output of transistor and the input of 
operational amplifier 512. 

Blank reference signal is formed by op amp 512 which 
buffers this signal to a DC voltage to generate the blank 

2 0 reference signal 514 which operates as a reference for 

the blank. The blank reference signal is combined with 
the VBI insert signal which has been elsewhere produced 
and is buffered by op amp circuit 516 and 518. The 
combined signal at node 520 is further buffered by 
25 circuits 522 and 524. These circuits are well known 
softening circuits which decrease the slopes of the 
voltage swings in order to avoid false color information. 

SampleBlank also feeds transistor 530 and its 
circuit. The SampleBlank turns on transistor 530 causing 

3 0 it to conduct and thereby bringing the clamped video 

signal 532 to its low potential. When SampleBlank is 
deasserted, transistor 530 is commanded to its high 
impedance state allowing the clamped video signal 532 to 
trc;ck the relative voltage swings of the video signal. 
3 5 For example, therefore, when the clamped video signal is 
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near 0 volts, the output video signal is black 
representing a zero bit in the user data. At some 
threshold above zero volts, say two volts, the video 
signal is encoded as white, representing a digital u l" in 
5 the user data. 

Circuit 534 forms a threshold detection circuit 
which compares the input signal 532 with a variable 
threshold which can be adjusted by potentiometer 536. 
The output of the potentiometer forms the VBI detect 538 

10 which is used by processor 554. 

The above has described the encoding operation. 
Much of this operation is common for the decoding. 

The receiver 144 and code stripper 14 6 are located 
at the video transmitting site, to allow the ADTAG 

15 information to be stripped off prior to transmission. 
Data transfer 152 is also affected by the circuit of 
Figs. 5A/5B. In view of the similarities of operation, 
the same circuit of Figs. 5A/5B is used to decode the 
information. This allows certain economies and saves on 

20 the cost of hardware. The circuit operates as either a 
decoder or an encoder based on a selection by a switch, a 
software switch, or an automatic detection of ADTAG 
information, for example. 

Fig. 7 shows a block diagram of the operation. The 

25 information is received by video receiver 154. Monitor 
700 includes the structure which strips codes and 
transfers data. The data is recorded by recorder 702 and 
also sent by modem 152 to the billing computer 162. The 
controlling console 704 enables certain systems to be 

30 appropriately controlled. 

As described above, the same system is used, but in 
this case controlled according to the flow diagram of 
Fig. 8. 

The flow diagram begins at step 800 by determining 
3 5 the beginning of a video frame. Using a similar 
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operation to that discussed above with respect to the 
encoding, the indication of vertical sync that is 
produced by processor 554 on its line 7 is used to 
indicate the beginning of the video frame at step 800. 
5 After that, video lines are counted at step 802 by 
counting each sync pulse. When the desired line is 
reached, this indicates the beginning of the VBI window. 
This line is then selected as a dummy at step 804 in 
order to remove the data from that signal . As described 

10 above, white dots are reconstructed as zeros and black 
dots are reconstructed as ones. This reconstruction is 
synchronized with a clock at step 806 in order to remove 
the data. The data is read at step 808 followed by 
switching back to the incoming video at step 810 to pass 

15 the rest of the frame unaltered. 

The data thereby stripped off at step 812 is sent 
via secured communications e.g. by encrypted modem 
information to the billing computer 162. The ADTAG 
database computer takes the information received in Chart 

20 2 ("Stripped Information*) and matches it to the 

information entered in Chart 1 ("Booking Information") . 
If items 1-4 of the booking information match, then items 
5-6 are approved and set up for payment. Items 1-4 are 
time, date, duration, TV station, Advertiser and tape 

25 information. Items 5 and 6 are price and ad agency. 

Payment is then processed by ADTAG by electronic transfer 
between the banks of the TV station, Ad agency and 
Advertiser . 

The most common errors that occurred are : 



Commercial didn't air for the correct length of ti 
(i.e. last 5 seconds were cut off). 

Commercial didn't air on the proper date and time. 
Stale tape was used . 
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These errors and additional errors that arise will 
be processed in a manner pre-established with all parties 
involved. Payment will be delayed until error is 
resolved according to these procedures. Some errors may 
5 fall in parameters that all entities have approved as 
acceptable and payment will not be delayed. 

For instance a 4 second cut-off may be acceptable 
and will be paid perhaps accordingly discounted. A 15- 
second cut-off is not acceptable and will be charged back 
10 to the TV station. All error processing will be listed 
out in contracts signed by all members of the ADTAG 
clearinghouse . 

In addition, errors where no airings occurred, but 
the booking remains in the ADTAG system will also be 
15 resolved on a basis to be established prior to entry to 
the ADTAG clearinghouse system. Therefore if a booking 
remains after the air date has passed the advertiser 
still has a reserve on its credit line that needs to be 
resolved and reversed. 

2 0 Security 

An important part of the ADTAG system is the use of 
security to avoid the possibility of fraud. Without 
security, the system could be fraudulently manipulated. 

As shown above, the teletext system provides a basic 

2 5 physical layer protocol for the data transmission. The 
transport and link layers must be specified in order to 
encode the data to ensure reliability and privacy. The 
system uses layering of error detection and correction 
mechanisms. For example the frames of data are encoded 

30 to guard against bad data at the receiver. The data can 
also be encrypted in block cipher mode using a 
periodically changing session key and public keying 
encryption algorithm. The details are shown in Fig. 4. 
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Fig . 9 shows a flow chart of operation which 
corresponds to the verification and authorization 
described above. Step 900 corresponds to the detecting 
of identification information within the commercial. As 
5 described above, this can correspond to obtaining 

information in a number of different ways, from a number 
of different places including but not limited to the 
vertical blanking interval and the closed captioning 
locations. Step 902 corresponds to a verification. This 

10 verification step can be carried out by operating in a 
secure premises. In those embodiments that do use a 
secure premises, the premises itself corresponds to the 
verification. Verification may include detecting the 
security of the encryption, correlating the commercial 

15 content, or comparing encryption in the vertical blanking 
with encryption in the closed captioning, for example. 
If verification is positive, a time and day stamp is 
added at step 904. This corresponds to a real time 
indication of when the commercial was actually broadcast. 

2 0 Step 906 begins the determination of whether the 

commercial was broadcast when it was supposed to. First, 
the commercial identification is compared against the 
database . 

At step 908, an indication of whether the 
25 identification matches the time and day when a schedule 
is determined. If so, the bank is authorized to pay the 
fee for the commercial, since everything was properly 
done as scheduled. 

If not, an error resolution system is determined at 
30 step 912. For each client, a separate error resolution 
system may be in place, or alternatively there could be a 
number of different error resolution systems which are 
applicable to different clients. The simplest error 
resolution system is that the commercial simply did not 
35 play. A more difficult scenario is when the commercial 
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did play, but played at the wrong time. A system can, 
for example, say that when the time differed by a certain 
amount, that the bill would be reduced by 50%. Other 
rules which are beyond the scope of this patent 
5 application could also be used. 

At step 914, the system determines that error 
resolution is not available, and kicks out the system for 
manual resolution at that point. 

Other modifications are within the disclosed 
10 embodiment. For example, other media including radio, 
cable, and others are contemplated. 
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What is claimed is: 

1. A method of automated billing of advertising, 
comprising : 

marking an advertisement with a code, said code 
5 indicating some aspect of the advertisement associated 
with billing; 

transmitting said advertisement over a channel; 
sensing said code on the advertisement; 
verifying security of said code; and 
10 obtaining billing information from the sensed code. 

2. A method as in claim 1, wherein said verifying 
security comprises verifying an encrypted signature on 
the code. 

3. A method as in claim 1, wherein said verifying 
15 security comprises receiving the advertisement at a 

secured premises and verifying a time when the 
advertisement was received along with the code from the 
advertisement at the secured premises. 

4. A method as in claim 1, wherein said marking 
2 0 comprises determining some aspect of the advertisement 

and encoding an indication indicative of said some aspect 
on the advertisement, and wherein said verifying security 
further comprises obtaining said code and said indication 
and using said code only if said verifying accurately 
25 describes the advertisement. 

5. A method as in claim 1, wherein said sensing 
and said verifying are carried out prior to said 

t ransmi 1 1 ing . 

6. A method as in claim 5, further comprising 
30 removing said code prior to said transmitting. 
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7. A method as in claim 1, wherein said sensing 
and said verifying are carried out after said 
transmitting. 

8. A method as in claim 7, wherein said marking 
5 comprises adding said code to each of a plurality of 

frames of the advertisement. 

9. A method as in claim 8, wherein said marking 
comprises adding said code to close captioning. 

10. A method as in claim 4 wherein said aspect is 
10 an average brightness of a portion of the image, and said 

marking further comprises placing an indicia indicative 
of said average brightness on said advertisement. 

11. A method as in claim 1 wherein said marking 
comprises placing said code in a vertical blanking 

15 interval. 

12. A method as in claim 1 wherein said marking 
comprises placing said code in a closed captioning 
portion. 

13. A method as in claim 1, further comprising 
20 taking an action which makes it more difficult for the 

advertisement to be automatically detected as an 
advertisement . 

14. A method as in claim 13, wherein said taking 
comprises removing said code prior to said transmitting. 



BNSDOCID: <GB 2331880A9_I_> 



- 31 - 

15. An automated advertisement billing system, 
comprising : 

a coder, which provides a code on an advertisement, 
said code not being seen as part of a picture forming 
5 program content, said code including information about 
the advertisement; and 

a code detection system, receiving said 
advertisement including said code, and determining a time 
when said advertisement is transmitted and providing an 
10 indication of the advertisement and the time; and 

a comparing device, comparing said indication with 
prestored information indicating a desired indication and 
time, and producing an authorization signal if said 
indication and time match said indication and time in 
15 said prestored information. 

16. A system as in claim 15, further comprising 
first means for making it more difficult to use said code 
to identify whether the part being played includes 
commercial or non- commercial content. 

20 17. A system as in claim 16, wherein said first 

means comprises a code stripper, located prior to 
transmission of the commercial, which removes said code 
and couples a removed code to said code detection system. 

18. A system as in claim 16, wherein said first 
25 means comprises means for adding a code to each frame. 

19. A system as in claim 15, wherein said coder 
comprises means for adding a code to all frames, some of 
said frames including information in said code indicative 
of the advertisement and others of said frames including 

30 information that is not indicative of the advertisement. 
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20. A system as in claim 15, further comprising 
second means for maintaining security of said code. 

21. A system as in claim 20, wherein said second 
means comprises operation at a secure premises. 

5 22 . A system as in claim 20, wherein said second 

means comprises third means for adding a correction 
indication to the code which relates to some content of 
the advertisement . 

23. A system as in claim 20, wherein said third 
10 means includes an integrator and said correlation 

indication indicates an average brightness of a portion 
of the advertisement. 

24 . A system as in claim 20, wherein said third 
means includes a closed captioning device and said 

15 correlation indication comprises a relationship between 
said code and content of closed captioning. 

25. A system as in claim 15, further comprising a 
billing computer, automatically producing an 
authorization to pay based on said authorization signal. 

20 26. A system as in claim 25, wherein said billing 

computer includes a set of rules which are used when said 
authorization signal is not produced, to provide 
automatic billing if said comparing device does not 
indicate said match and parameters of mismatch are 

25 described by said rules. 
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27. A method of automatically billing for a 
commodity which is produced during a time slot and is 
time slot sensitive, comprising: 

producing said commodity to be shown during said 
5 time slot; 

producing a first code which is associated with said 
commodity, said code indicating identity of said 
commodity; 

determining some aspect of content of said 
10 commodity, and producing a second code indicative of said 
aspect ; 

associating said first code and said second code 
with said commodity ; 

receiving said content in a way which indicates that 
15 said commodity was actually produced at a time of 

receiving, and obtaining said first and second codes 
therefrom; and, 

determining from said first and second codes 
received by said receiving whether the content matches 
20 the second code and validating the first code only if so. 

28. A method as in claim 27, further comprising 
billing for said commodity only when said first code is 
validated . 

29. A method as in claim 28, further comprising 
25 taking an action that makes it less likely that said 

codes can be used to automatically detect said commodity. 



BNSDOCID: <GB 233188QA9_I_> 



- 34 - 

30. A method of automatically billing for 
advertising , comprising : 

marking an advertisement with a code, said code 
indicating some aspect of the advertisement that is 
5 associated with billing; 

processing said advertisement in a way which makes 
it more difficult for a user from detecting codes on only 
advertisements and not on other program content, in order 
to prevent the user from automatically detecting presence 
10 of an advertisement; and 

obtaining billing information from said code on said 
advertisement when said advertisement is actually 
transmitted . 

31. A method as in claim 30, wherein said 
15 processing comprises stripping said code prior to 

transmission. 

32. A method as in claim 30, wherein said 
processing comprises marking all frames of all programs 
with a code. 

20 33. A method as in claim 30, further comprising 

determining if a code represents information or 
represents non- information, and billing for the 
advertising if the code represents said information. 

34. A method as in claim 30, wherein said 

25 preventing comprises placing readable information in a 
closed captioning portion. 

35. A method as in claim 30, further comprising 
verifying a security of the code. 
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36. A method as in claim 30, wherein said 
prevention comprises encoding said code on frames of said 
program using a steganographic technique. 
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